Toponym Resolution: A First Large-Scale Comparative Evaluation

Toponym resolution (TR) is the task of mapping the name of a location to a spatial representation of the location referred to, such as the centroid of the location, given as latitude/longitude. While a number of systems for automating the task have been described in the literature, to date no comparative evaluation study has existed, mainly for lack of a standard benchmark (i.e., gazetteer and evaluation corpus). On the basis of a benchmark methodology and dataset, we present the first systematic account of the utility of different heuristics for the toponym resolution task, based on experimental comparison on two novel, large-scale gold-standard corpora. Each heuristic’s utility is evaluated in isolation, and in addition, two previously reported complex methods are replicated in full.